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ABSTRACT 

One of 15 core modules in a 22-module st. .es designed 
to train vocational education curriculum specialists (VECf.. this 
guide is intended for use by both instructor and student in a variety 
of education environments, including independent study, team 
teaching, seminars, and workshops, as well as in more conventional 
classroom settings. The guide has five major sections. Part 1/ 
Organization and Administration, contains an overview and rationale, 
educational goals and performance objectives, recommended learning 
materials, and suggested reference materials- Part II, Content and 
Study Activities, contains the content outline arranged by goals. 
Study activities for each goal and its corresponding objectives 
follow each section of the content outline. Content focus is on 
educational evaluation in retrospect (historical antecedents), 
terminology used in educational evaluation, the education and role of 
the evaluation specialist, the different conceptions of educational 
evaluation and the purposes for which it is conducted, and criteria 
used in evaluating vocational education programs and instruction. 
Part III, Group and Classroom Av-tivities, suggests classroom or group 
activities and discussions kc ed to specific content in the outline 
and to specific materials in the list of references. Part IV, Student 
Self-Check, contains questions directly related to the goals and 
objectives of the module, which may oe used as a pretest or postte£:t. 
Part V, Appendix, contains suggested responses to the study 
activities frcn part II and responses to the student self-checks- 
(HD) 
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PREFACE 



Uho is a vocational eoucation curriculum specialist? The answer 
to this questiOf) is not as simple as it might af.oear. A vocational 
education cu'^ricjlum specialist is likely to work in many different 
capacities, includin9, but not limited to: instructor, department 
chairperson, dean of vocational -techn ical education, vocational super- 
visor, orincipal, state or local director of vocational education, and 
curricul un] coordinator. 

The specidlist is, oerhaps, more identifiable by his/her respon- 
sibilities, which include, but are not limited to: 

• planning, organizing, actualizing, and controlling the work 
of an educational team performed to determine and achieve 
objectives. 

t planning, organizing, and evaluating content and learning 
processes into sequential activities that facilitate the 
achievement of objectives. 

• diagnosing present and projected training needs of business, 
industry, educational institutions, and the learner. 

• knowing, comparing, and analyzing different theories of curric- 
ulum development, management, and evaluation and adapting them 
for use in vocational -technical education. 

This teaching/ 1 earning module is part of a set of materials repre- 
senting a comprehensive curriculum development project dealing with the 
training of vocational education curriculum specialists. The purpose 
of this two-year project was 1) to design, develop, and evaluate an^ 
advanced-level training program, with necessary instructional materials 
based on identified vocational education curriculum specialist compe- 
tencies, and 2) to create an installation guide to assist instructors 
and administrators in the implementation process. 

The curriculum presented here is, above all else, designed for 
flexible installation. These materials are not meant to be used only 
in the manner of an ordinary textbook. The materials can be used 
effectively by both instructor and student in a variety of educa- 
tional environments, including independent study, team teaching, 
seminars, and workshops, a*", well as in more conventional classroom 
settings . 

Dr. James A. Dunn 
Principal Investigator and 
presently Director, 
Developmental Systems Group 
American Institutes for Research 
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PART 1 

ORGANIZATION AND ADMINISTRATION 



Guidelines 

This study guide has five major sections. Each section contains useful 
information, suggestions, and/or activities that assist in the achievement 
of the competencies of a Vocational Education Curriculum Specialist, Each 
major section is briefly described below. 

L'^^^L. _lL ORGANIZATION AND ADMINISTRATION 

PART I contains an Overview and Rationale, Educational Goals and Performance 
Objectives, Recommended Learning Materials, and Suggested Reference 
Materials. This section will help the user answer the following questions: 

• How is the TOdule organized? 

• What is the educational purpose of the module? 

• What specifically should the user learn from this module? 

• What are the specific competencies emphasized in this module? 

• What learning materials are necessary? 

• What related reference materials would be helpful? 

PART_ n_: CONTENT AND STUDY ACTIVITIES 

Part II contains the content outline arranged by goals. The outline is a 
synthesis of information from many sources related to the major topics 
(goals and objectives) of the module. Stuc'y activities for each goal and 
its corresponding objectives follow eech soction of the content outline, 
allowing students to complete the exercises related to Goal 1 before going 
on to Goal 2, 

PART HI: GROUP AND CLASSROOM ACTIVITIES 

Jhe '7'Cti vi ties-Resources" column in the content outline contains refer- 
ences to classroom or group activities and discussion questions related to 
specific content in the outline. These activities and discussion questions 
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are located in PART III and are for ni<i iorial use of either the instructor 
or the student. Both the cl ass-Mi),;: a. :,-;es and disci.ss-.on questions are 
accompanied by suggested respon^^ '"or u-^.- as helpful e aiiiples only--they 
do not represent conclusive ans.. to it:^- problems and 'ssues addressed. 
Also contained in the "Acti vities-i- mrces" column are the reference 
numbers of the resources used to develop the content outline. These 
reference numbers correspond to the numbers of the Suggested Reference 
Materials in PART I. 

PART I_V: STUDENT SELF-CHECK 

PART IV contains questions directly related to the goals and objectives of 
the module. The self-check may be used as a pre-test or as a post-test, 
or as a periodic self-check for students in determining their own piogress 
throughout the module. 

PART^V^ APPENDICES 

Appendix A contains responses to the Study Activities from PART II, a-;d 
Appendix B contains responses to the Student Self-Check. The responses 
provide immediate feedback to the user and allow the module to be used 
more effectively for individualized study. They have been included in the 
last part of the module as appendices to facilitate their removal should 
the user wish to use them at a later time rather than concurrently with 
the rest of the module. 

Approximately 30 hours of out-of-class study will be necessary to complete 
thi s module. 

Overview and Rationale 

In the curriculum development process, the evaluation component 
directly and continuously affects all other activities. This module 
is designed to provide the vocational educator with an overview of the 
development of contemporary evaluation theories and familiarize him 
with the terms used in educational evaluation. Formal, systematic 
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educational evaluation is defined and differentiated from standard 
educational research, evaluation research, and inforrral nonsystematic 
evaluation. The most popular currer^ evaluation theories are briefly 
described, and the roles that eval fill in impl ementin^j those 

theories are explained. 

The module is divided into two major content areao. The first area is 
an overview of the forces that have influenced the educational evalu- 
ation movement since the scientific fiianagement movement in the early 
part of the twentieth century. 1\\ ---volution of educational evaluation 
terminology is presented, and contemporary definitions of those terins 
are explained. In addition, the differences in purpose between educa- 
tional resoar nd educational evaluation are pointed out. Also, 
formal, systematic educational evaluation is differentiated from in- 
formal, everyday, nonsystematic evaluation, and ttie purposes of the 
two types of evaluation are delin(?ated. 

Tfie second major content area is an overvi(?w of current evaluation ttieory. 
Major ^'valuation ifiodels are briefly described and ttieir salient [joints 
outlined. Tfie roles tfiat evaluators [jlay in ttu; various KiodfMs and ttie 
skills and knowh.'dgf.* rf^juired of ttiefii are also describ(»d. 



Goals and Objactives 

Upon coffiplot ion of this fnodule Iho student v/ill bo dblo to dctiiovo tfio 
follov/infj "-jodls and objoctivos: 



GOAL 13.1: Bl. AWARt OF IHL SOCIAL, POLITICAL, AND ECONOMIC FACTORS TflAT 
IIAVL CONTRIBUTFD TO TUF OrvrLORMFNT OF Tflt FDUCATIONAL FVAL- 
UAflON MOVFMFNT. 

(Jljjr.'Ctivo 13.11 IrJontify the iiid.jof fjctivities thdt have been 
cti(j rdctr?ri /erl fjs (jfiucat iona 1 evaluation in 
tfie [jast. 

OI)jr'(.t i vr* 13. IX Identify the major (wents and factors of 

ttif» last two decades tfiat fiave had the ii.>,t 
(jrorirwriced r?ff(H.t on ttie educational tjvalu- 

<\ t ' ')ri IIIOVeilKMlt . 

f.OAl 13.?: P>I lAMIMAR Willi 111! IIRMINOLOGY IJM.l) IN I.DUCAi I ONAl 
1 VAI DA HON. 

Objr*(tivr* 13.^1 Off inr* f 'fluca t i una 1 evalutjtion as it is 
[)resf/nt iy conceij , ja 1 l /.ifd . 

Ohjf'ctivf* 13.?;^ 0 i St iri()ij I sti aiiionf) terms (such as researcti, 
iiifasurrMitent, and t.(?stinfj) that ar(? commonly 
intr»r*cfianf)»'d with the term revaluation but 
wfiicfi dcj not actually mean ttie same. 

f/)AI n. {: l!l AWARl 01 Illi MMIl ARI IIlS AND IJI f I FKINCI S HI IWI I N I DIJCA- 
IIONAI Rl M ARCH ANO I I)IJ(,A1 lONAl IVALIJAIION. 

Ohjffj. iv*» 13.31 I /plain ttie rlif Terences in tffchniques and 

purposes between fflucat iona I res(»arcti and 

r'flur.at ioria I eV'i I U'itlori. 
ObjfMtiv'^ I 1.3? Oi St iri'pjl sti betwerTi t^yorytUy educa t if)ri»j I 

f'V.i I uat ion activities and systeiiujti': f'duca- 

I )r)tial <'va lu.itlori. 
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Objective 13.33 Distinguish among activities that are 
characterized as grading, ir.easurement , 
research, and evaluation. 



GOAL 13.4: BE AWARE OF THE QUALITIES AND KNOWLEDGE THAT EDUCATIONAL 
EVALUATORS MUST POSSESS AND THE ROLES THEY MUST PLAY. 

Objective 13.41 Identify decision situations that require 
an evaluator's expertise. 

Objective 13.42 Identify the three major roles that eval- 
uation specialists are required to fill in 
a decision-making context. 

Objective lA-.^l Identify the knowledge and skills that 

evaluation specialists must have and use 
in their various roles . 



GOAL 13.b: BE AWARE OF THE DIFFERENT "CONCEPTIONS OF EDUCATIONAL 

EVALUATION AND THE PURPOSES FOR WHICH IT IS CONDUCTED. 

Objective 13.51 Identify the four major types or con- 
ceptions of educa tional evaluation. 

Objective 13.52 Distinguish among the purposes of the four 
general types of evaluation. 

Objective 13.53 List the characteristics that are common 

to dll four types of educational evaluation. 



GOAL 13.6: BL AWARE OF THE CRITERIA USED IN A DECISION-MAKING CONTEXT 
TO EVALUATE VOCATIONAL EDUCATION PROGRAMS AND INSTRUCTIONAL 
INILRVCNTIONS. 

ObjocLive 13.6,1 List criteria that are commonly used in 
evaluations of vocational education. 

Objective 13.62 Distinguish between the criteria used to 
evaluate vocational education and those 
used to evaluate education in general. 
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PART II 

CONTENT AND STUDY ACTIVITIES 



Goal 13.i 



Content Outline 



Activities-Resources 





\\\\\\\\\^^\^ 

Goal 13.1: Be Aware of the Social, 
Political, and Economic Factors That Have 
Contributed to the Development of the 
Educational Evaluation Movement 



A . Educational Evaluation in Retrospect* 

1. Evaluation is an activity we all engage in 
every day. It usually follows a rationale 
and includes criteria for making a decision. 

2. According to Alkin and Fitz-Gibbon, evaluation 
is usually comparative, that is, one option 

or course of action is examined and compared 
with another (2). 

3. Daily evaluations are usually informal and not 
systematic. 

B. Historical Antecedents to Educational Evaluation 

1. Early scholars evaluated the effects they had 
on their students, but their evaluation 
"reports" were confidential. Evaluation was 
done for personal reasons, but it was, never- 
theless, done to improve instruction. 

2. The ientific management movement in educa- 
tional administration provided the impetus for 

20 



* See the Glossary of 
Terms at the begin- 
ning of Part III. 



(2) "Methods and 

Theories of Eval 
uating Programs . 



Content Outline (continued) 



the first comprehensive evaluations of educa- 
tion. However, the methods were rather un- 
scientific, and the purpose was largely for 
economic reasons rather than for instructional 
improvement (10) . 

The Eight-Year Study was the first formal, 

systematic evaluation that used sophisticated 

techniques. Its impact, however, was minimal 

because of World War II (11). 

During the Civil Rights Movement of the 1960s, 

the "separate is not equal" doctrine provided 

more reason for evaluation. 

"Life adjustment" education critics demanded 

proof that schools were effective (5). 

Sputnik created another issue in the "schools 

are soft" controversy (5), (11). 

The Elementary and Secondary Education Act of 

1965 focused attention on evaluation as the 

means for determining the effectiveness of 

educational programs. 

The public accountability movement gave 

further impetus to systematic evaluation in 

determining program, school, and teacher 

effectiveness (24). 

Decentralization has increased the need for 
district level evaluation units to tailor eval 
uation to local performance objectives.* 



(10) Behavioral 

Science and Edu- 
cational Adminis- 



tration , 
III, IV. 



Chaps, 



(11) Lh_^_ Curriculum: 
Retrospect and 
Prospect , pp. 
26-44. 



( 5 ) The Transforma- 
tion of the 
School , pp. 
338-347. 

(5) Same as above, 
p. 347. 

(11 ) Same as above, 
p. 91. 



( 24 ) Educationa l Eval- 
uatio n: Theory 
and Practice, p. 91 



* See Discussion 
Questions A and B in 
Part III. 
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'\ r*-;f'iiAc'.'C utc followiru; activities. 

1. The call for educational evaluation, whether by individual 
teachers or by society, has been based on a desire either to 
improve the quality of student learning or to justify the 
expenditure of money on the educational enterprise. For example 
in the time of Plato the need for educational evaluation was 
based on Plato's desire to improve the quality of his students. 
For the decades shown below, indicate what was behind the push 
for educational eval uation--a desire for a better student learni 
(quality), or a desire for better cost-effectiveness (economy)? 

a. 1910-1920 

b. 1930-1940 

c. 1950-1960 

d. 1960-1970 

e. 1970-present 

2. What was the first major federal U?gir,lation that required eval- 
uation of the funded programs? 

3. What three factors seem to be most prominent in the push for 
"accountability" of the schools today? 

(!)(!(: Apijcndix A for poiuyihlc a?isb)ep.i, ) 



Study Activiti es 
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Goals 13.2, 13.3 



Content Outline 



^;sS:;s\Goal 13.2: Be Familiar with the 
^XnN Terminology Used in Educational 
Eval uation 






A, Educational Eval uati on : An Examination of 
Terminol ogy 

1. Because educational evaluation has only 
recently been recognized as a discipline, 
there is often confusion and ambiguity in its 
terminology. 

2. Evaluation is defined most popularly as a 
systeiiiatic , formal process for identifying 
and collecting data on educational phenomena 
to as^^ist in the decision-making process (17), 
(18), (14), (11), (2), (19). 

a. The term "educational phenomena" may 
include programs, curricula, instructional 
interventions, or other observable facts 
or events in the educational process. 

b. Formal judgment by the evaluator is not 
part of tho evaluation process. 



Activities-Resources 



(17) "The Methodology 
of Evaluation," 
pp. 39-83, 

(18) "The Countenance 
of Educational 
Evaluation," pp. 
523-540. 

(14) Discrepancy 
Evaluation . 

(11) The Curri cul um : 
Retrospect and 
Prospect . 

(2) "Methods and 

Theories of Eval- 
uating Programs ," 
pp. 2-3. 

(19) Educational Eval- 
uation and 
Decision Making . 



23 



-13- 



Content Outline (continued) 



Mea sureme nt is not the same as evaluation 
(24). 

Grading is not the same as evaluation. 
Grading schemes or systems can be 
eval uated . 

Evaluations are conducted to provide infor- 
mation for those experiencing or demanding 
'^iccountabi 1 i ty ( 15) . 

irch and evaluation both use many of 
;Mf same techniques, but the ultimate pur- 
poses are different (11). 

(1) Researchers are looking for conclu- 
sions: evaluators are looking for 
worth . 

(2) Researchers are concerned with the 
generalizabil ity of their discoveries; 
evaluators are concerned with one 
speci f ic si tuation . 

Two distinctive types of evaluation that 
were first described by Michael Scriven in 
1967 are "formative" and "summative" 
eval uations . 

(1) Formative evaluations are those con- 
ducted for the immediate improvement 
of a program that is still modifiable. 
This type of evaluation provides 
decision-making inforriitition to the 
developer or manager of a project or 
program. 



(24) Educational 



Eval uation : 
Theory and Prac- 
tice, Chap. 2. 



(15) 



"Accountabil ity 
A Sobering 
Thought," p. 21 



(11) The Curriculum : 
Retrospect and 
Prospect, Chap. 
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Content Outline (continued) 



(2) Summative evaluations are those 

connected with the final judgment of 
a project or program. This type of 
evaluation provides decision-making 
information to the us^^r or consumer 
of a product or proi.: : (17).* 



(17) "The Methodology 
of Evaluation," 
pp. 39-83. 

Students should 
complete Classroom 
Activity 1 which 
relates to Goal 
13.3. See Part III, 
See Discussion 
Questions C, D 
and E. 
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B. Study Activi ties 



1. Define "systematic educational evaluation." 

2. Define "formative" and "summdtive" evaluation, and indicate the 
differences between the two, 

3. Indicate whether or not each of the following activities should be 
characterized as systematic educational evaluation. If not, indi- 
cate how the activity might best be described, using such terms 

as "measurement," "grading," "research," and so on. 

a. A data processing instructor wants to establish the manual dex- 
terity level of prospective students relative co a specific set 
of criteria, so she administers a series of manual dexterity 
examinations to all incoming students, 

b. The dean of vocational education at a community college 
conducts a pretest-post-test study to see if programmed 
instruction is preferable to the traditional instructor-student 
relationship. 

c. Mr. Kriebelmeir, the carpentry teacher, keeps a log of "points" 
that his students earn in the performance of daily classroom 
activities so he can prepare defensible reports on their 
progress at the end of the term. 

d. A district curriculum specialist compares the results of several 
different approaches to individualizing instruction in order to 
see which of them should be adopted districtwide. 

e. A local college professor attempts to determine if the students 
in vocational education classes perform better and learn more if 
they are financially rewarded for their efforts in class. 

f. A state law requires that all teachers be appraised at least 
semi-annually in terms ui their demonstrated effect on students. 



i a'fiLl arVj t".rjyb: i 
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1. List several similarities between research and evaluation. 

2. How do the purposes of educational research and educational 
evaluation differ? 

3. Are the results of educa' .^nal evaluations general izable? Why? 

4. How does measurement differ from evaluation? 
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Goal 13.4 



Content Outline 




A . Edu cation of the Eval uation Special ist 

1. Decision situations determine the role(s) 
that the evaluator will have to fill. 
According to Stuff 1 ebeam et_ al_. , there are 
four categories of decision situations: 

a. choosing among optional and sometimes 
competing goal s; 

b. choosing one of several optional methods 
of achievi ng a goal ; 

c. choosing among the alternatives involved 
in implementing a program; and 

d. choosing to terminate, revise, or continue 
a program (19). 

2. The first three categories listed above are 
the initial focus of the evaluator in the 
"interface" role. 

3. In the "technical" role, the evaluator pro- 
duces information to serve these first three 
decision situations. 

4. In the "administrative" role, the evaluator 
plans " ^valuation and coordinating 

acti 



Activities-Resources 



(19) Educational 

Eval uation and 
Decision Making , 
p. 297. 



* See Discussion 
Question F in 
Part III. 
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B. Study Activities 

r.ii'tr : 1':.:^^ t^lir S . out ^ o,}tA iViu additions!. 

>^.' 'V>\7:. v»7 ^ rr>rplctr \lu> fnidi-Duuj activities. 

1. Stufflebeam ard his associates divide the role of the evaluation 
specialist into three categories. List the three categories. 

2. The authors of "Evaluator Role" take a slightly different tack 
from Stufflebeam and his associates in defining the roles of the 
evaluator. Are the two conceptions of "role" compatible? If so, 
place the roles described in "Evaluator Role" into the categories 
that Stufflebeam and his associates delineated. 

3. List four abilities or types of knowledge that evaluators must 
have in order to play the interface role effectively. 

4. List five abil ities that the evaluator must have in order to 
effectively fill the role of evaluation technician. 

5. From where do educational evaluators often "borrow" techniques to 
use in their role as evaluation technicians? 

6. List three activities that the evaluator performs in the adminis- 
trative role. 

7. Stufflebeam and his associates group the skills and knowledge 
needed by the evaluation specialist into four general categories. 
List those four categories and include several examples from each. 

8. When describing the interface role of the e, jator, Stufflebeam 
and his associates list several criteria that are commonly used 
in the decision-making process. List those criteria, and give an 
example of each from vocational education. 
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Goal 13.5 



Content Outline 



vv^Goal 13.5: Be Aware of the Different 




ERIC 



A . Evaluation Theorie s for Quality Education : An 
Overview 

1. Bobbitt and Charters, curriculum reformers of 
the 1920s, produced curricula that were 
highly acceptable to school administrators and 
others because their content was quantifiable 
and measurable (5), (11).* 

2. Evaluation theorists have various conceptions 
of the purpose of evaluation and have created 
evaluation "models" that correspond to these: 

a. goal -attaining models (21); 

b. judgmental models emphasizing intrinsic 
criteria (12); 

c. judgmental models emphasizing extrinsic 
criteria (12), (18); and 

d. decision-facilitation models (2), (14), 
(19), (20), (24).* 

B. Evaluating Evaluation 

1. Humanistic educators have questioned the very 
motives of evaluation; they have concluded 
that evaluation is often used as a selection 
mechanism, as an untenable application of 
industrial mechanisms to human behavior, and 
as part of a bureaucratic record-keeping 
structure (9). 

-21- 



Activities-Resources 



(5) The_ Transformation 
of the School > p. 
199. 

(11) The Curricul um : 
Retrospect and 
Prospect , Chap. 1. 

* See Discussion 
Question G in Part 
III. 

(21) Basic Principles 
of Curriculum and 
Instruction. 



(12) Educational Eval 



(18) 



uation : New Roles , 
New Means ■ 
"The Countenance of 
Educational Evalu- 
ation," pp. 523-540. 
(2) "Methods and 

Theories of Evalu- 
ating Programs." 
(14) Discrepan cy Evaju.- 
ation . 

(19) Educational Evalu- 
ation and Decision 



Making" . 

(20) "Evaluation as 
Enlightenment for 
Decision-Making." 

(24) Educational Eval- 
uation : Theory and 
Practice , pp. 210- 
215, gives an excel- 
lent comparison of 
selected character- 
istics of the various 
model s 

* See Discussion 
Question G in Part 
III. 

(9) "An Evaluation of 
Evaluation," pp. 
3-14. 



Content Outline (continued) 



2. other critics feel that evaluation is a (4) 
danger to an evolving understanding of the 
learning process (4) . 

3. Some educators feel thdt evaluation has been 
used negatively, rather than for improvement. 

Quality Control and Evaluation 

1. Evaluation and quality control are not the 
same . 

2. A quality control system incorporates all the 
expressed purposes of evaluation; goal 
attainment (and examination), self-examination, 
information for decisions, and judgment. 

3. Quality control essentially maintains consis- 
tency in output with respect to the standards 
or criteria of acceptance; evaluation seeks to 
collect information relevant for decisions 
leading to improvement. 



"Educational 
Accountability 
from a Human- 
i Stic 

Perspective," 
pp. 19-21. 
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study Acti vi ties 



The major reason an organization is in business is either to produce a 
profit 0'- to provide a service. Schools produce profits in the form 
of people who have the requisite skills to fill necessary roles in 
society. In order to be effective and efficient in the conduct of the 
educational enterprise, educators select objectives, define the methods 
and conditions required to me^t those objectives, and maintain quality 
control by evaluation. Vocational educators were among the first to 
adopt methods and techniques from the business model, perhaps because 
the content of vocational education was so visibly related to business. 

Cremin (5, p. 199) has written that the techniques for curriculum de- 
velopment advanced by Bobbitt and Charters in the early 1920s were suc- 
cessful and were accepted so heartily because they were able to quantify 
the content of curricula and to measure it. They were able to evaluate 
the worth of what was being taught and to demonstrate the efficiency of 
thei r methods. 

Trends in education in the late 1920s and the 1930s showed somewhat less 
emphasis on measurable, quantifiable objectives and more on reordering 
society. However, the work of Ralph Tyler (21), formulated in the 1930s 
and 1940s, serves as the basis of one form of educational evaluation 
currently in use. Tyler felt that the bases for curriculum development 
should be: selecting objectives, selecting learning experiences, orga- 
nizing the learning experiences, and evaluating learning. 

Campbell and Gregg (3) assert that the general purpose of evaluation is 
to improve the effectiveness of goal achievement. Popham (13) insists 
that the purpose of evaluation is the formal assessment of the worth of 
educational phenomena. Hammond (14) feels that the purpose of evaluation 
in education is to discover whether innovation is effective in achieving 
expressed objectives, while Stufflebeam (19) and Alkin (1) argue that 
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evaluation should be used to facilitate decision-making. Tyler, Metfessel, 
and Michael (21) declare that the purpose of evaluation is to learn 
whether or not expressed objectives are met. Scriven (17) claims that 
the assessment of merits is the most important purpose of evaluation. 



There are as many processes for educational evaluation as there are 
expressed purposes. However, the four general processes, or models, 
are: (1) the goal -attaining model; (2) the judgmental model emphasizing 
intrinsic criteria; (3) the judgmental model emphasizing extrinsic 
criteria; and (4) the decision-facilitation model. 



Campbell and Gregg (3) have delineated four points or four steps that 
seem to be part of all the different processes. The first step in any 
evaluation process is selecting and defining the particular phase of 
the activity to be evaluated. Once one knows what is going to be eval- 
uated, criteria or basic assumptions upon which interpretations or 
judgments will be based can be developed. The remaining steps in the 
evaluation process are collecting data pertinent to the criteria, inter- 
preting and analyzing that data, and drawing conclusions. 

The goal-attainment models, as exemplified by Tyler, Metfessel, Michael 
and Hammond (14), first decide on a goal, then express the goal in 
behavioral terms, and finally assess the degree to which the goal is 
attained. Goal -attainment models spring from the Eight-Year Study of 
progressive education, a study guided in part by Ralph Tyler. Metfessel, 
Michael, and Hammond have expanded the original Tyler model, but the 
emphasis remains goal attainment. 

The most common example of a judgmental model emphasizing intrinsic 
criteria is the accreditation model. Worthen and Sanders (24) point out 
that the purpose of the accreditation model is to identify deficiencies 
in the education of teachers and students relevant to content and pro- 
cedures . Accrediting agencies are often not as interested (publicly at 
least) in the worth of what is being taught as they are in the methods 
used to teach it and the facilities available. Personal judgment (and 
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bias) by professional colleagues often serves as the basis for evaluation 
in the accreditation model. 

Scriven (17) and Stake (18) are the chief proponents of the judgmental 
models emphasizing extrinsic criteria. These models are concerned with 
the effects of the educational process. Scriven is responsible for the 
formative-summative distinction common to evaluation today. He has also 
recommended that the worth of the goals should be determined before an 
evaluator decides how well they have been achieved. Once an evaluation 
has been undertaken, the emphasis, according to Scriven, should be on 
the effects of the program. Scriven also argues that it is the respon- 
sibility of the evaluator to make comparisons with competing processes 
in order to facilitate any decision on the part of the operators of 
the program. The end result of the evaluation process in his opinion 
would then be the improvement of the instructional process. 

Stake's countenance model distinguishes between the descriptive and the 
judgmental acts of the evaluator according to what he perceives as 
throe phases of an educational program: what happens before the program 
(antecedents), what happens during the program (transactions), and what 
happens as a result of the program (outcomes). He then makes a distinc- 
tion between what was intended as part of the program and what was 
actually observed. Judgments based on absolute criteria (standards) 
and relative criteria (personal judgment) may then be made. As with 
Scriven's model, the purpose underlying the countenance model is the 
improvement of instruction. 

The decision-facilitation models, as characterized by Alkin (1), Provus 
(14), and Stufflebeam (19), are less concerned with determining the 
worth of educational goals than with collecting and presenting data 
to decision-makers who then determine the worth or make a value judgment. 
Stufflebeam's model is called the CIPP model after Contract, Input, Pro- 
cess, and Product. The CIPP process is primarily concerned with delin- 
eating the questions raised by decision-makers, obtaining data relative 
to those questions, and synthesizing the information for use by the 
d^'^ision-maker. 
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Malcom Provus (14) devised a model for evaluation based on the premise 
that evaluation is the comparison of performance to standards. He calls 
his model the Discrepancy Model because it pays particular attention to 
the discrepancies between what the program operators claim are their 
standards for student performance and students' actual performances. 
The Discrepancy Model requires an independent evaluator whose chief pur- 
pose is to aid program improvement and counsel program administration. 
Again, the Discrepancy Model is a decision-facilitation model: a service 
to program operators. 

The Alkin, or CSE Model (1), is probably the most lauded of the decision- 
facilitation models. It is composed of three phases: pre-formative, 
formative, and summative. The pre-formative phase has two activities 
in which the evaluator is involved: needs assessment and program plan- 
ning. The formative phase contains an implementation evaluation and a 
progress evaluation. The summative phase is an outcome evaluation and 
provides data to the decision-maker that aids in the determination of 
whether to continue or discontinue the program. The CSE Model is the 
only model that includes evaluation as a part of the total process of 
program planning, development, operation, revision, and continuance or 
discontinuance. Unfortunately, most programs have already been put into 
effect by the time the evaluation component is introduced, so the CSE 
Model is difficult to employ in its entirety. 

James Macdonald (9) has questioned the motives behind the rapid growth 
of educational evaluation in recent years. He feels that the justifi- 
cation for evaluation offered by most evaluators--that of concern for 
finding out what has been learned or accomplished in order to improve 
the process in the future--is secondary to the continuance of the fund- 
ing of most projects or programs. He accuses many evaluators (or those 
calling for evaluation) of being concerned not with learning, but with 
selection. Evaluation is seen by Macdon^.ld as often being a part of the 
record-keeping fetish of the bureaucratic structure and an application 
of industrial activity to educational programming. Eval-.iation becomes 
a way of separating means from ends, and it is all done "scientifically" 
because evaluation is defined as a technical problem. 
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Macdonald (19) and Arthur Combs (4) both feel that the current evaluation 
emphasis is dangerous to the humanistic image of man and to an evolving 
understanding of the learning process. Macdonald feels that the danger 
is heightened by federal grant pronrams (such as Title I), by the 
national testing programs, by accountability movements, and by the in- 
dustrial focus of the schools. He proposes that evaluation procedures 
be built around the following ideas: 

(1) goals are merely hypotheses that may evolve or change; 

(2) intuition is real and results in unpredictable activity; 

(3) inconsistency between goals and behavior may well signify 
experimentation and productive change; 

(4) careful sequential activity over time may be an enemy of 
personal growth; and 

(5) learning can be viewed as a way of changing one's past, 
experiences by reinterpreting them in light of new actions 
and consequences (9, p. 14). 

Macdonald's and Comb's criticisms are valid and familiar to curriculum 
specialists. They call for an examination of the applications of eval- 
uation within the education system. 

A glance at the literature of vocational education reveals a plethora 
of books, articles, and studies on evaluation, accountability, cost- 
effectiveness, quality control, and research. Most of these documents 
have been published since provisions in the Vocational Education Amend- 
ments of 1968 mandated periodic evaluation of the vocational programs 
supported by federal funds. Many approaches have been used and recom- 
mended to assure quality vocational programs including evaluation that 
focuses on providing data for decision-makers, evaluation for judging 
whether a program is good or bad, and evaluation for self-assessment. 
The intent of the clauses in the 1968 VEA mandating periodic evaluation 
was to increase or maintain quality vocational education, necessitating 
an emphasis on both the process and the product of the vocational edu- 
cation system. 
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Educators are again employing a term borrowed from business and industry 
to name the process of ensuring that the best possible education is pro- 
vided: quality control. Quality control implies more than just evalu- 
ation or accountability. A quality control system does not limit the 
process to providing data for decision-makers, or to providing an oppor- 
tunity for self-examination, or to placing a value label on a program. 
A quality control system does all of these things at all phases of the 
vocational program, and it includes everything and everyone involved in 
the vocational education experience. A viable quality control system is, 
at the same time, both a reflection of planning and an integral component 
of planning, in that it serves as a method for determining if and how 
well programs are accomplishing their stated objectives and, at the 
same time, serves as an input to the needs assessment phase of vocati'Mal 
planning. 

A quality control system is not only concerned with the attributes of 
the "product" of the process, but with the methods employed in the pro- 
cess, the equipment and facilities used, the personnel, and the planning 
and operation of the system itself. A quality control system assures 
vocational education that is current, relevant, and of high quality. It 
is the mechanism by which all components of the process are strengthened 
through the use of evaluation. 

1. Evaluation models or theories may usually be categorized according 
to purpose. List the four major categories of evaluation models 
and state the purpose for each. 

2. Although the four categories of evaluation models have different 
purposes, what activities appear to be a part of all the processes? 

3. What are the basic differences in purpose among the goal-attainment 
models, the judgmental models, and the decision-facilitation models? 

4. What are the five major steps of evaluation that the evaluator is 
involved in when the CSE model is fully implemented? 
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James Macdonald and Arthur Combs have been two of the most vocal 
critics of the evaluation movement in education. List several of 
their criticisms. 

Macdonald offered five assumptions around which evaluations should 
be built. Examine those areas, and then in one or two sentences 
summarize his fears of what evaluation could produce. 

Is quality control the same as evaluation? Why? 
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Goal 13.6 



Content Outline 



Goal 13.6: Be Aware of the Criteria 
Used in a Decision-Making Context to 
Evaluate Vocational Education Programs 
;^S^and Instructional Interventions. 



Evaluation Criteria * 

1. Wenrich and Wenrich identified the five most 
common (and time tested) criteria for voca- 
tional education evaluations: 

a. program completion rates; 

b. student competency; 

c. cost-efficiency of the program (6); 

d. placement of students; and 

e. employer-employee satisfaction (23). 



Activities-Resources 



* Students should 
complete Classroom 
Activity 3 which 
relates to Goal 
13.6 in Part III. 
Also see Discussion 
Question H. 



(6) "The Role of 
Vocational Edu- 
cation in 
Improving Ski 11 s 
and Earning Capa- 
city in the State 
of Ohio: A Cost- 
Benefit Study." 

(23) Leadership ">n 

Administration of 
Vocational and 
Technical Educa- 
tion , pp. 267-272. 
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B. Study Acti vi ties 



Based on youy reading of the content outline and any additional 
references as suggested^ conrplete the following activities, 

1. Wenrich and Wenrich identify five criteria that they consider 
critical to the success of any vocational program evaluation. 
List these five criteria, and ni\ oxnmples of methods or 
techniques that an evaluatic^ specialist might use to collect 
data to determine if the cr'ierid are being met. This exercise 
requires outside reading in the Recom^ rnded Materials. 

Wrapup Activity 

NOTE; To meet the basic requirerrents of this module, select one of the 
following activities and complete it as directed. Each of the letters 
under Activity 1 identifies on activity. If you wish to gain additional 
credit beyond the basic requirements, you may choose a second activity 
to complete. Consult with your instructor first if you wish additional 
credit. 

1. The purpose of each of the activities below is to give you an oppor- 
tunity to determine how local educational agencies are implementing 
the evaluation activities required by the Vocational Education Amend- 
ments of 1968. Select a high school district near you and interview 
several members of the vocational education hierarchy to determine 
answers to one of the following questions or group of questions. 
Summarize the answers you receive in a four- to five-page paper. 

a. How does the district indicate to the State Advisory Council 
the extent to which consideration was given by the district to 
the findings and reconmendations of the most recent advisory 
council evaluation submitted to the U. S. Office of Education? 
(It would help to have a copy of that report.) 
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b. Does the district evaluate the effectiveness of the funds pro- 
vided under VEA? How? 

c. Does the district evaluate the effectiveness of the instruction 
vocational students receive in terms of job placement, earnings 
relative to students who have not received training, or earnings 
relative to students who received training at another level of 
the education system, such as community college? How? If not 
in terms of any of these, how do they evaluate their programs? 
Who conducts the evaluation? 

d. What form of evaluation is used by the district, that is, 
which of the "models" is employed, if any, to evaluate the 
vocational curricula? (You will have to be familiar with all 
four evaluation models.) 

0. How much of the district's vocational education budget, or for 
that matter total instructional budget, is used for evaluation 
activities? 

Using the resources found in the Suggested References section, study 
one of the four evaluation models. Defend its use as a viable 
evaluation model for vocational education in a four- to five-page 
paper. 
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Group and Classroom Activities 



42 



PART III 

GROUP AND CLASSROOM ACTIVITIES 
Classroom Activities 

GLOSSARY 

The following terms a -fi.ied in order to clarify their meanings in the 
context of this module. 



CIPP . An acronym formed from the first letters of the four basic kinds 
of evaluation under the decision-facilitation model as advocated by 
Stufflebeam. The letters represent context, input, process, and pro- 
duct. 

COMPARATIVE EVALUATION . Evaluation in which one process/product is com- 
pared with another or with others in terms of effectiveness, cost, 
acceptance, etc. 

CRITERION . A standard for judging and validating. 

EDUCATIONAL PHENOMENA . Observable objects, facts, events, or processes 
in the educational setting that are evaluated. 

F ORMATIVE EVALUATION . Evaluation that is concerned with program improve- 
ment, ana that generally requires evaluator intervention in the program 
or process. 

OBJECTIVE . A point of accomplishment that can be verified within a given 
time and under specifiable conditions, which, if attained, reflects 
progress toward achievement of a corresponding goal. Objectives are 
sometimes known as behavioral objectives, performance objectives, and 
terminal objectives . 

PRE-TEST . 

POST-TEST . A component of an evaluation system or research design that 
requires testing before and after instruction or implementation of a 
program to determine the extent of student progress or achievement. 

RELIABILITY . A term usually applied to measurement instruments, indicat- 
ing that the instrument is consistent in the way it measures qualities 
or characteristics. 
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SUMMATIVE EVALUATION . Evaluation concerned with determining overall pro- 
gram effectiveness; it generally requires little or no evaluator inter- 
vention in the program or process. 

SYSTEMATI C EVALU ATION. A systematic, formal process of identifying and 
collecting information on educational phenomena to assist decision- 
makers in choosing among available decision options. 

VALIDITY . A term usually applied to measurement instruments, indicating 
that the instruments adequately cover what they are designed to cover, 
that they correlate wi ch the factors or traits they are designed to 
measure, and th'iz they correlate with other measures of the same 
trait. 

NOTE: The following activities are designed for use in the classroom to 
stimulate discussion on specific topics covered in this module. The 
activities are designed to be used following student self-study; however, 
depending on the background and abilities of students, these 
activities may not require previous study. All classroom activities 
are keyed to the content outline to indicate an appropriate point 
at which they might be presented. 

1. Using the following situation and related questions, establish 
several groups in the class and allow the students to brainstorm 
possible solutions to the problem. Have each group of students 
choose either the formative or summative evaluation role, and after 
their brainstorming session, have them describe the procedures they 
believe appropriate to carrying out that evaluation. 

SITUATION: Assume that you are members of an evaluation consulting 
firm called in to help plan a procram for formative and summative 
evaluation of a U.S. Office of Education project in which the cliof 
focus is developing curriculum materials for use in training vocd- 
tional education curriculum specialists. The materials (consisting 
largely of self-instructional booklets) are being developed at con- 
siderable cost to the taxpayers, and the question is how they may 
be most effectively used, and, having used them as effectively as 
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possible, whether they are really worth the cost. Choose either the 
formative or summative evaluation role, and describe the procedures 
you believe are appropriate to carrying out that evaluation. 

RELATED QUESTIONS : The following questions are those that evaluators 
might ask. They are not in any specific order, and no clue is given 
as to whether the formative or summative evaluator would ask the 
question. The class should discuss whether or not these questions 
relate to formative or summative evaluations. 

a. What are the objectives of the project? Are they measurable? 

b. Is a pretest of any kind applicable? 

c. May a posttest be used? 

d. Is there a market for vocational curriculum specialists? 

e. Are there other programs in existence that could be adapted to 
the objectives at less cost? 

f. What kinds of tests may be used to determine if the project 
objectives are met? 

g. Will there be opportunities to pretest the materials developed 
prior to actual use? 

h. What criteria constitute "effective"? 

i. What are minimal proficiency levels for vocational curriculum 
special ists? 

j. What size of a group of learners should be used in the testing 
of the materials? 

k. What decision criteria have the project personnel specified? 

Using the situation described in Activity 1, divide the class into 
four groups and have each group assume the role of evaluators 
whose purpose is one of the following four: (1) goal attainment; 
(2) judgmental based on intrinsic criteria; (3) judgmental based 
on extrinsic criteria; or (4) decision facilitation. Each group 
should develop lists of criteria that it would examine for its 
specific purpose in evaluation. (The lists of criteria developed 
should be clearly consistent with each of the purposes; i.e., 
goal attainment, etc.) 
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Campbell and Gregg (3) have identified several points that they 
consider to be part of all the different evaluation processes. 
Discuss those points in the context of the four types of evaluation 
and attempt to substantiate their assertion. How does the purpose 
of the evaluation affect the way in which those activities are 
conducted? You might attempt to chart the different points against 
each of the four evaluation types in a 4 x 4 matrix such as the one 
below: 

Evaluation Purposes : 



ol 

Cl.1 



CT/j 



CL 

e 



Goal 
Attainment 



1 . Select and define 
the phase of the 
activity to be 
evaluated 



Develop or define 
eval jati ve 
criteria 



Collect data 



Intf?rpret data 
and draw conclu- 
sions 



The entire 
process and/ 
or program 



Were or are 
goal s met? 



Judgmental 
Based on 
Intrinsic 
Criteria 



All data will 
be relative 
to attain- 
ment of 
goal s 



The goals 
are either 
met or they 
aren' t 
yes = good 
no = bad 



Number of 
books in the 
1 ibrary 



Judgmental 
Based on 
Extrinsic 
Cri teria 



Decision 
Facil itation 



All phases 
of the plan- 
ning, devel- 
opment 3.nd 
operation of 
a program 



Did the 
students get 
jobs? 
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Activities for Additional Credit 



NOTE: These activities are designed for the student who wishes t ob- 
tain credit beyond the basic requirements of the module. You ray ^hnose 
to write a paper on one of these activities, or discuss 
the activity with the instructor, or you may select some other method 
to complete the activity. 

1. Select a vocational program you know of or one from a nearby high 
school and determine, from the director or from the principal, by 
what criteria the vocational program was evaluated in their last 
accreditation evaluation. 

2. Develop an evaluation plan for a proposed new textbook, curriculum 
package, or instructional intervention in the vocational subject 
area with which you are most familiar. 

3. Select one of the evaluation criterion categories identified by 
Wenrich and Wenrich (see Recommended Materials #3, pp. 267-269) and 
delineate metho^^ of collecting data for decision makers relative to that 
criterion category. 

4. Perform a library search for materials relevant to the evaluation 
of vocational ec^ 'Crition during one of the decades from 1920 to the 
present. Prepare an annotated bibliography of all materials that 
are discovered. T'le annotation should include a short summary of 
the criteria uS'* n the evaluation process during that decade. 
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Discussion Questions 



A. What are some examples of everyday evaluations we all make? What 

criteria do we consider in making these evaluations? For what purpose 
are everyday evaluations conducted? 

(Examples of everyday evaluations might include: 

a. color of tie or scarf to wear with shirt and suit or blouse; 

b. the fit of one's clothing; 

r. degree of danger in driving on freeways. 

The criteria for these everyday evaluations are: 

a. hues and tones of clothing and their coordination; 

b. conformance to body contours, tightness, or looseness; 

c. one's driving experience, degree of congestion of the freeways. 

Purposes of everyday evaluations: Most everyday evaluations are 
conducted for the purposes of improving one's appearance, increasing 
comfort, improving the use of time, and in some cases, preserving 
one's safety.) 



B. What are some common, everyday evaluations that a vocational teacher 
might make? What criteria might be considered and how are these 
criteria unique to vocational education? For what purposes might 
these evaluations be conducted? 

(Some everyday evaluations that a vocational teacher might carry out 
are the: 

a. relative safety or cleanliness of the classroom; 

b. effectiveness of a particular type of instruction in a given 
situation; 

c. type of material to use in a classroom. 
The criteria for these evaluations include: 

a. amount of clutter around work areas, number of students, students' 
attitude ^ toward safety; 

b. number oi tudents present, the degree of complexity of the 
concept or process being taught; 

c. expense of materials, the type of materials used in the actual 
work environment, the danger involved in the use of certain 
mriterial s . 

The £!irpose^ of these evaluations: Most everyday evaluations in 
vocationaT education are for the purposes of improving the immediate 
instruction, ensuring safety, or making the learning situation as 
realistic as possible. ) 
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What are some systematic evaluations common to school settings? 

(Examples might include field tests of texts or instructional 
materials prior to adoption.) 



What roles do research and measurement play in the development of 
instructional materials? 

(Research and measurement may be used to determine what, why, and how 
students can learn.) 



What role does formative evaluation play in the development of 
instructional materials? Summative evaluation? 

(Formative evaluation is concerned with improving instructional 
materials. Summative evaluation is concerned with judging curriculum 
materials aS effective or not, primarily to arrive at a decision to 
replicate or adopt them.) 



What part do evaluators' personalities or demeanor play in the way 
they are perceived by the people for whom the evalui'.tion is being con- 
ducted? Those being evaluated? 

(The evaluator who has a "know-it-all" attitude will likely antagonize 
both those who are being evaluated and those for whom the evaluation 
is being conducted. The overly aggressive evaluator may give the 
impression that he is biased in some way toward the program or project, 
thereby creating an unfavorable attitude in the two groups. The 
evaluator must maintain an unbiased stance when collecting data upon 
w.''ch decisions will be based. To do otherwise would jeopardize the 
reliability of the evaluation results, and may unfairly or incorrectly 
bias readers of the evaluation report for or against a particular 
project or program.) 



What social movement created the types of educational administrators 
who would heartily accept a curriculum that wa?: so "scientific" it was 
measurable? What social factors led to the cre,»,v'on of that nwvement? 
(10) 

(The movement in the early part of the 20th certurv for efficiency in 
business [the scientific nia.iagement movement] c:\^^t^-/ :; probable 
climate for the use of "measurable," "scientific" cv ^vili) . Some of 
the social factors that led to this movement 1. .:lu^*-r:' r :, rniiiigration, 
mass migration from farms or the rural South to r^eation 
of large city school systems, and gross ineff/cit conduct 
and management of private and public organizatior^al act ; v , rie:. . ) 
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What other criteria might be used in evaluating vocational programs? 
What are those used in other educational evaluations? (19) 

(Other criteria might include such things as student satisfaction and 
the adaptability of students to changes in the occupation or job. 

Criteria used in other educational evaluations include the ability 
of the students to grasp abstract concepts and to read and write 
efficiently. These same criteria also apply to many vocational 
evaluations- ) 
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Part IV: 



Student Self-Check 
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PART IV 
STUDENT SELF-CHECK 



GOAL 13.1 

1. The evaluation of educational efforts has been a part of the formal 
educational process for as long as we h'ive had one, but the evalua- 
tion effort wasn't systematized until rather recently. List four 
functions performed by educators prior to 1950 that were then charac- 
terized as evaluation. (13.11) 

2. List four major social events or movements that have occurred in the 
last two decades that have contributed to the educational evaluation 
movement. (13.12) 

GOAL 13.2 

3. Define educational evalurition as it is most popularly conceptualized 
and employed. (13.21 ) 
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Match the terms in the left column with the definitions in the right 
column. Put the letter of each definition in the space next to its 
corresponding term. (13.22) 



summative evaluation 

measurement 

grading 

research 

formative evaluation 
educational evaluation 
_ judging 
accountabil i ty 



a. Assessing the merit of students' 
intellectual attributes or 
accompl i shments 

b. The process of requesting,^ 
requiring, or providing evidence 
that schools are giving society 
its money's worth of education 

c. The act of assessing worth 

d. The act or process of determining 
overall program effectiveness 

e. The process of developing a model 
or theory that identifies all 
relevant variables in an environ- 
ment and hypothesizes about their 
relationship. 

f. The act or process of providing 
decision information to program 
developers or managers 

g. The process of identifying and 
collecting data on educational 
phenomena to assist decision- 
makers in choosing among available 
options 

h. The act of assigning numbers to 
objects and events according to 
some predetermined criteria 



GOAL 13.3 

5 



Explain the basic differences in purpose between educational 
evaluation and educational research. (13.31) 
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Indicate how each of the activities depicted below might best be 
described or characterized as measurement, research, grading, non- 
systematic evaluation, systematic evaluation, formative evaluation, 
or summative evaluation. (13.32, 13.33) 

A local trade union consortium has complained 

that new apprentices who have had vocational training in high school 
cannot perform many of the basic tasks required of them, so the 
district vocational education director mandates that performance 
examinations be given to all vocational program graduates. 

Instructors of the vo'-ntional programs in 

th¥ "a bovT district are required to base their final assessment of 
their students on the results of the performance examinations. 

The teacher of a medical laboratory technician 

l^rse^'~a'fTer~unch deliberation on the merits of various brands and 
styles of laj coats, decides that it would be best if all ^-^Mdents 
wore orange lab coats without pockets while in class. 

An educational psychologist feels that the 

color of lab coats has an effect on students' attitudes toward safety 
and responsibility in the classroom, so he examines matched pairs of 
students in different classrooms where the students wear different 
colored coats and compares their attitudes. 

A building trades teacher feels that if 

students were allowed and encouraged to form their own construction 
company, they would learn better and faster. The company is formed 
and students assume the various positions of a construction company 
hierarchy. They then bid for jobs in the community around the school. 
The supervisor of vocational education is impressed with the opera- 
tion of the program, but after it has been in operation for two 
semesters, the school board questions whether the increase in learning 
(if any) is worth the added risks of having the students work off 
campus. A consultant is called in to help the supervisor decide 
whether- or not to allow the continued operation of the company. 

A college professor is asked to supervise 

an occupational needs assessment for the state division of vocational 
education so the state can decide which of its vocational program 
objectives should be emphasized. The division wants the needs assess- 
ment conducted independently so they can better determine how to 
direct and allocate funds for the various vocational programs. 
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A textbook company, dismayed at their lack 
oFsuVc^sTTnTiaTit'^^^^^ a market in newly released books after an 
initial spurt of sales, decides t.j try out all new materials prior 
to final printing and tlien juchie whether or not they should De 
al tered . 

GOAL 13.4 

Place a check marK (/) next to those deci:.ion situations listed 
•low that require an evaluator's expertise. (13.41) 

a. Is one program better than aiiother? 

b. Is the better program worth the added expense of conducting 
it? 

c. Should the school board buy nonunion products to use i^ 

vocational classes? 

d. Six teachers who appear to have equal qualifications apply 
for one job opening. Who should get the job? 

8. What are the three major roles that evaluators are required to 
assume in educational evaluation? 

9. List two skills or pieces of knowledge that an evaluator must 
possess in order to be effective in each of the above roles. (13.43) 

GOAL 13.5 

10. List four major "conceptions" or types of evaluation, and state 
the purpose of each. (13.51, 13.52) 

11. What four activities are common to all types of evaluation? (13.53) 
GOAL 1 3.6 

12. List five criteria that are commonly employed in vocational education 
eval uati ons . (1 3.61 ) 
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Place a check mark (/) next to those criteria listed below that 
would most likely be used in evaluating vocational programs. (13 

a. St^Jents' ability to find work upon graduation 

b. Students' attitudes toward work 
c. Students' demonstrated competence 

d. Students' preferences in work clothes 

e. Teacher morale 

f. Postgraduate earnings of students 

g. Work experience of the teaching staff ^ 

n. Demonstrated competence of the students in oral and 

written communication skills 
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Part Vs 



^ Appendices 
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PART V 
APPENDICES 

Appendix A: 

P ossible Study Activity Responses 

The answers below are designed to give you feedback on the exercises in 
this guide. If correct answers exist to given questions, they are provided. 
In the case of more complex exercises, sample responses or suggestions are 
provided. 



GOAL 13.1 

1 . a . economic 

b. quality 

c. quality 

d. economic 

e. primarily economic, but quality is included 



2. Elementary and Secondary Education Act of 1965 



3. a. decrease in local support 

b. increase in federal support 

c. decentral ization 



GOAL 13.2 

1. "Systematic educational evaluation" is a systematic, formal process of 
identifying and collecting information on educational phenomena to 
assist decision-makers in choosing among available decision options. 



2. Formative evaluation is that type of evaluation concerned with program 
improvement. 

Summative evaluation is that type of evaluation concerned with deter- 
mining overall program effectiveness. 

Formative evaluation is a service to the developer or manager of a 
program or project or curriculum material. Summative evaluation is a 
service to the user of the phenomena. 
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3. a . No; measurement 

b. Yes; systematic evaluation 

c. No; grading 

d. Yes; systematic evaluation 

e. No; pure research 

f. Yes; (hopefully) systematic evaluation 



GOAL 13.3 

1. Both research and evaluation measure. 
Both collect data. 

Both use statistical techniques to analyze data. 
Both employ technological tools. 
Both prepare formal reports. 

2. The purpose of research is to reach conclusions; the ultimate 
purpose of evaluation is to determine worth of phenomena. 



3. No. The results of educational evaluations are not general i zabl e. 
Evaluations deal with one specific program or phenomena, and there 
is no intent to generalize. What works in one situation may or may 
not work in another, and no attempt is made to find out. 



4. Measurement is conducted to determine position on a scale, while 
evaluation is conducted to determine worth relative to specific 
cri teria . 



GOAL 13.4 

1 . a . interface role 

b. technical role 

c. administrative role 



59 



52- 




Yes; the two conceptions of role are compatible. "Evaluator Role" 
describes skills that Stuff lebeam and his associates included in 
their categories. 

The interface role includes the evaluator as defendant and the evaluat 
as attorney. The technical role includes the evaluator as expert wit- 
ness, and the evaluator as court reporter. The administrative role 
includes the evaluator as judge and juror. The roles of evaluator as 
detective and social worker are roles that aren't recognized as accep- 
table, but they do exist, and would probably best be placed in the 
interface role. 



Consider your answer correct if it includes any four of the following: 

a. ability to spell out the ends to be served by the evaluation 

b. knowledge of available options or alternatives 

c. ability to determine criteria to be used in the decision-making 
process 

d. ability to identify audiences for the reporting of evaluation results 

e. ability to interact with (if not conduct) the technical aspect of 
the evaluation 

f. ability to determine the information needs of each audience of the 
evaluation , 

g. ability to prepare an evaluation report that will meet the audience s 

needs 



Consider your answer correct if it includes any five of the following: 

a. converting criterion statements into measures 

b. sampling 

c. measurment 

d. data processing' 

e. data analysis 

f. data interpretation 

g. data reporting 



Educational evaluators often ''borrow" techniques to use in their 
role as evaluation technicians from educational research methodology. 



Consider your answer correct if it includes any three of the following: 

a. deciding on the focus of the evaluation 

b. planning the evaluation and scheduling the activities 

c. coordinating the various roles of the evaluator 

d. appraisal of techniques/methods employed 

e. record keeping 



60 

-53- 



a Knowledge of the concep_tiOii of evaluation: functions, inputs, 

bTfpTiYs,' sequence, " environment, physical catalysts, and human 

<\c)ents and their roles 
b Knowledge rec^uired in perJlorming evjJ^Ja tjoji woii; : focusing the 

evil lia'ti on," 'information collection and organization, data 

analysis and reporting, and administration 

c. Knowledi]j2 of the evaluation _s_ettjj£: the education system and 
all social ,"poTitical , and economic aspects 

d Other areas: economics (especially for cost-benefit evaluations), 
poTitical'science, and general systems theory 

a. tinie--thG length of a course in instructional hours governs the 
number of learning units that can be accomplished _ _ 

b nionPv--funds allocated for instructional support are crucial in 
terms of s'jpplies, audio-visual materials, text and reference 
books, field trips, ratio of students to teacher, equipment 
availability and maintenance, etc. 

c egjioment avai 1 abi 1 i ty--need to use outdated equipment 

d. achievement rriteria-a trainee-graduate must meet entry-level 
employment standards since these are the minimum criteria for 

employment . ^ . • 4.u ^.^Q^tQcf 

e atM^udes--the achievement of the affective domain is the greatest 
■ challenge faced by the instructor and is the primary expressed 
need of business and industry . 
f political factors-include conflicts between career aspirations 
of minorities and the job opportunities available; the closed 
shop and the maintenance of a restr-tive labor supp y by some 
unions; the dichotomy between parent and student goals 



GOAL 13.5 



to determine if the program or project met its 



Goal - atta ining : 
objecti ves 

Judgmental emphasizing intHjislc g^Xexia^ ^ - /^u. 

d^1TrTo"nt^nt~a7fdrprocedures and decide whether they (the con- 
tent and procedures) are good or bad 



to identify deficien- 



oudgme ntaj emphasi zing extrinsic criteria : 
cies in tlie process and judge the worth 
Decision- facilitation : to provide information 
wW the ultimate end being the improvement of 
product 



to identify deficien- 



for decision making 
the process and 
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2. a. selecting and defining what is to be evaluated 

b. establishing or determining evaluation criteria 

c. collecting data 

d. analyzing data and drawing conclusions 



3. Goal-attainment checKS only to see if "'hat was set out to be accom- 
plished was in fact accomplished. No value is placed on goals or on 
whether or not they were reached. Judgmental models place a value on 
the goals, the process, and the product. Improvement can come only 
with the next program. Decision-facilitation models intend to improve 
the product and the process by being involved through the total system. 
Decision-facilitation models are an aid in decision-making, but no 
judgment is passed as to value or worth. 



4 . a . needs assessment 

b. program planning 

c. implementation evaluation 

d. progress evaluation 

e. outcome evaluation (suinmative) 



5. a. evaluation is dehumanizing 

b. involves bureaucratic record keeping 

c. stifles creativity 

d. evaluation is really for selection, not improvement 



6. Locking into set procedures promotes conformity 

Inconsistency doesn't necessarily mean no growth has occurred 



7. No. Quality control is not the same as evaluation. Evaluation is one 
mechanism of a quality control system. A truly comprehensive quality 
control system would employ evaluations of all four types. 



GOAL 13.6 

1. a. program completion: checking school records, exit interviews 

b. competency: competency exams, licensing exams, etc. 

c. cost efficiency: cost-benefit studies, checking other comparable 
programs for their costs 

d. employment placement: follow-up studies 

e. employee-employer satisfaction: follow-up studies 

(The examples above are just a few of many. Examples that are, in 
your judgment, comparable to or better than those above are acceptable 
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Appendix B: 

Possible Self-Check Respo 'e^ 
GOAL 13.1 

1. The evaluation of educationa' f ; )rts l is oe'^n a part cf the formal 
educational process for as loriq as \;e have had one, but the eval- 
uation effort wasn't systematized until rather recent'y. List four 
functions performed by educators orior to 1950 that w**:re then char- 
acterized as evaluation. (13.11) 

- qradinq students based on daily classwork 

- personal appraisal of students by teachers 

- teacher's self-appraisals 

- testing students 



2. List four major social events or movements that have occurred in 
the last two decades that have contributed to the educational eval- 
uation movement. (13.12) 

(Consider the student answer correct if it includes any four 
of the fol lowina. ) 

- the civil rights movement 

- federal legislation mandating evaluation 

- increase in federal funds 

- decrease in local funds 

- school decentralization movement 

- public accountability mover* nt (consumerism) 

- genuine desire on the oart of educators to improve the 
educational orocess 



GOAL 13.2 



3. Define educational evaluation as it i$ most popularly conceptualized 
• and employed. (13.21 ) 

Educational evaluation is a systematic, formal orocess of 
identifying and collecting information (or data) on educational 
phenomena to assist decision-makers in choosina among available 
decision options. 
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Match the terms in the left colunm with the definitions in the 
riqht column. Put the letter of each definition in the space next 

to its corresoondi nq term. (13.22) 



suniinative evaluation 



measureinen-i 



qradi ng 



research 



formative evaluation 



educational evaluation 



judqinq 



accountability 



a. Assessina the merit of students' 
intellectual attributes or 
accomplishments 

b. The process of requesting, 
requiring, or providing evidence 
that schools are qiving society 
its money's worth of education 

c. The act of assessing worth 

d. The act or process of determining 
overall orogram effectiveness 

e. The process of developing a model 
or theory that identifies all 
relevant variables in an environ- 
ment and hyoothesizes about their 
relationship 

f . The act or process of providing 
decision information to program 
developers or managers 

g. The process of identifying and 
collectinq data on educational 
Phenomena to assist decision- 
makers in choosing among available 
options 

h. The act of assigning numbers to 
objects and events according to 
predetermined criteria 



GOAL 13.3 



5. Explain the oasic differences in purpose between educational 
evaluation and educational research. (13.31) 

An acceptable answer would describe evaluation as being 
conducted to improve the education process and to determine 
worth. Research has as its purpose reaching generalizable 
conclusions. Evaluation is not generalizable. 



64 



EKLC 



-58- 



Indicate how each of the activities deoicted below mioht best be 
described or characterized as measurement, research, grading, non- 
systematic evaluation, systematic evaluatio.., formative evaluation, 
or sunimative evaluation. (13.32 , 1 3.33) 

measuremen t A local trade union consortium has complained that 
new apprentices who have had vocational trainina in high school 
cannot perform many of the basic tasks required of them, so the 
district vocational education director man^intes that performance 
examinations be given to all vocational [) \cram Graduates. 

grading Instructors of the vocational orocrams in the above 
district are required to base their final a^^essment of their 
students on the results of the performance '^xami. tions. 

everyday evaluatio n (n onsys tema tic) The teacher of medical 
laboratory technician course, after much deliberation on the merits 
of various brands and styles of lab coats, decides that it would be 
best if all students wore oranoe lab coats without pockets while 
in class. 

research An educational psychologist feels that the color of lab 
"coats has an effect on students' attitudes toward safety and resoon- 
sibility in the classroom, so he compares the attitudes of niatched 
students in different classrooms, where the students wear ditterenr 
colored coats. 

summativc. evaluatio n A building trades teacher feels that if stu- 
dents were allowed and encouraged to form their own construction 
company, they would learn better and faster. The company is formed 
and students assume the various positions of a construction company 
hierarchy. They then bid for jobs in the community around the^ 
school. The supervisor of vocational education is impressed with 
the operation of the program, but after it has been in operation for 
two semesters, the school board guestions whether the increase in 
learning (if any) is worth the added risks of havina the students 
work off-camPus in competition with local construction companies. 
A consultant is called in to hel: ^.he supervisor decide whether or 
not to allow the continued operation of the company. 

systematic evaluation (decision-facilitation; A college professor 
is asked to supervise an occupational needs assessment for the state 
division of vocational education so the state can decide which of 
its vocational program objectives should be emphasized. The division 
wants the needs assessment conducted independently so they can better 
determine how to direct and allocate funds for the various vocational 
programs. 
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for mative evaluation A textbook company, dismayed at their 
Tack of success in maintaining a market in newly released books 
after an initial spurt of sales, decides to try out all new mater- 
ials prior to final printing and then judae whether or not they 
should be altered. 



GOAL 13.4 



7. Place a check mark (i/) next to those decision situations listed 
below that require an evaluator's expertise. (13.41) 

y a. Is one program better than another? 

^ b, Is the better program worth the added expense of 
conducting it? 

c. Should the school board buy nonunion products to 
use in vocational classes? 

d. Six teachers who appear to have equal qualifications 
apply for one job opening. Who should get the job? 



8. What are the three major roles that evaluators are required to 
assume in educational evaluation? (13.42) 

a. interface role 

b. technical ro'e 

c. administrative role 



9. List two skills or pieces of knowledge that an evaluator must 

possess in order to be effective in eacli of the above roles. (13.43) 

(Consider the student answer correct if it includes any two of 
the following skills or knowledge for each role.) 
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a. Interface Role 



(1) ability to spell out the ends to be served by the evaluation 

(2) knowledge of available options or alternatives 

(3) ability to dGtern;iine criteria to be used in the decision- 
making process 

(4) ability to identify audiences for the reporting of evalu- 
ation results 

(5) ability to interact with (if not conduct) the technical 
aspect of the evaluation 

(6) ability to determine the information needs of each audience 
of the evaluation 

(7) ability to prepare an evaluation report that will meet the 
audience's needs 

b. Technical Role 

(1) converting criterion statements into measures 

(2) sampling 

(3) measurement 

(4) data processina 

(5) data analysis 

(6) data interpretation 

(7) data reporting 

c. Administrative Role 

(1) deciding on the focus of the evaluation 

(2) planning the evaluation and scheduling the activities 

(3) coordinating the various roles of the evaluator 

(4) appraisal of techniques/methods employed 

(5) record keeping 



GOAL 13.5 



10. List below four major "conceptions" or types of evaluation, and 
state the purpose of each. (13.51, 13.52) 

a. goal attainment which purpose is: determination of goal 
achievement 

b. .iudqmental emphasizing intrinsic criteria which purpose is: 
judgment 

c. judgment emphasizing external criteria which purpose is: 
judgment 

d. decision-facilitation which purpose is: information for decision- 
making 
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11. What four activities are common to all types of evaluation? (13.53) 

a. Selecting and defining what is to be evaluated 

b. Establishing or determining evaluation criteria 

c. Collecting data 

d. Analyzing data and drawing conclusions 
GOAL 13.6 

12. List five criteria that are commonly employed in vocational 
education evaluations. (13. bl) 

Answers here should list either the five major categories- 
Program Completicn, Competency, Cost Efficiency, Employment 
Placement, Employee-Employer Sotisfaction--or specific 
examples of them. 

13. Place a check mark (v/^) next to those ^rit.^ria listed below that 
would most likely be used in evaluatin,] vocational programs. (13.62) 





a. 


Students' ability tc find wo: n upon graduation 




b. 


Students' attitudes toward work 




c. 


Students' demonstrated competence 




d. 


Students' preferences in work clothes 




e. 


Teacher morale 




f . 


Postgraduate earnings of students 




g. 


Work experience of the teaching staff 




h. 


Demonstrated competence of the students in oral and 
written comi'^nni cation skills 
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CIPCE^ Attitude Scale No. 1 .4 



Attitudes toward EA^^^iJorAl ^J^ALl^JiPi] • ^^^^^ ^ number of state- 
ments about the evaluation of educational programs. A program can be a 
lesson, a course, a whole curriculum, or any training activity. Consider 
each statement as a statement of opinion. If you agree at least a little 
bit with the statement, circle the letter A. If >ou disagree even a 
little bit with the statement, circle the >:tter D. If you both agree 
and disaqree, ')r if ynu liave no opinion, leave the letters uncircled. 

A = AGREE D - DISAGRL: Blank = Neither 

1. AD The major purpose of an educational evaluation study should 

be to gather information that will be helpful to the educators. 

J. A D It is important for tlie program evaluator to find out how well 
variou<^ people like the program. 

3. A D Generally speaking, an educational program should be evaluated 

with reference to one or more "coiarol" programs. 

4. A D The evaluator should accept the responsibility of finding the 

strongest, most defensible, and publicly attractive points of 
the program. 

5. A D In evaluating a program, it is at least as important to study 

and report on the types of teaching as it is to study and re- 
port on the amount of learning. 

6. A D The ovaluator should draw a conclusion as to whether or not 

the goals of the program are worthwhile. 

7. A D It is more important to evaluate a program in comparison to 

what other programs do than to evaluate it with reference to 
what its objectives say it should do. 

a. A D Principals and superintendents should not gather data about 
the quality of instruction in the classroom. 

9. A [) The task of putting educational objectives into writing is 
more tho responsil 1 i ty of the evaluator than that of the 
educator. 

10. A D II is essential that the full array of educational objectives 
be tated before the program begins. 
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HAD Evaluation studies would improve if they gathered more 

kinds of information, even if at the expense of gathering 
less reliable information. 

12. A D Evaluators should ignore data that cannot be objectively 

veri f ied. 

13. A D Education should have more of an engineering orientation 

than it now has. 

MAD The job of an evaluator is mostly one of finding out how 
well students learn what they are supposed to learn. 

15. A D Evaluation should aid an educator in revising his goals even 
while the program is in progress. 

16 A D The process of decision-making about the curriculum is one of 

the weakest links in the present operation of the schools. 

17 A D Educators have some important aims that cannotbe stated 

adequately by anyone in terms of student behaviors. 

18. A D Information from an evaluation study is not worth the trou- 

ble it makes . 

19. A D The first job in instruction is the formulation of a state- 

ment of objectives. 

20. A ^ A teacher should tell his students any and all of his teach- 

ing Oi)jectives. 

21. AD The major purpose of educational evaluation is to find out 

the worth of what is happening. 

22. A D The evaluator should be a facilitator more than a critic 

or reformer or scholar. 

23 A ; ome school experiences are desirable because theyround 

out a child's life-whether or not they increase his com- 
petence or change his attitudes. 

24 A D An evaluator should find out if the teaching is in fact tne 

kind that the school faculty expects it to be. 

2b A D Whether or not an evaluation report is any good should be 
decided pretty much on f'O same grounds tho^ research jour- 
nal editors use to decide whether or not a manuscript should 
be published. 



ERIC 
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^ 26. A D The main purpose of evaluation is to gain understanding of the 
causes of good instruction. 

27. A D Description and value judgment are equally important compo- 

nents of evaluation. 

28. A D In conducting an evaluation, there is no justification for the 

exercise of subjective judgment of any kind by the evaluator. 

29. A D Educational evaluation is a necessary step in the everyday 

operation of the school. 

30. A D The strategy of evaluation should be chosen primarily in terms 

of the particular needs the sponsors have for evaluation data. 

31. A D The educational evaluator should attempt to conceal all of his ' 

personal judgment of the worth of the program he is evaluating. 

32. A D The sponsor of an evaluation should have the final say-so 

in choosing or eliminating variables to be studied. 

33. A D The main purpose of educational evaluation is to find out what 

methods of instruction work for different learning situations. 

34. A D Parents' attitudes should be measured as part of the evalua- 
^ tion of school programs. 

35. A D An evaluator finds it almost impossible to do his job without 

intruding upon the operation of the program at le<7st a little. 

36. A D All important educational aims can be expressed in terms of 

student behaviors. 

37. A D Some educational goals are best expressed in terms of teacher 

behaviors . 

38. A D It is essential that evaluation studies be designed so that 

the findings are general i/:able to other curricula. 

39. A D An evaluation study should pay less attention to the statisti- 

cal significance of a finding than an instructional research 
study would. 

40. A D Evaluation interferes with the running of schools more than 

i t helps. 

41. A IJ Little evaluation planning can be done before you get a state- 

ment of instructional objectives. 

4;^ A U Th(> loader of an evaluation team should be a teacher. 



ERIC 



-65- 



A D The entire school day and the entire school experience 

should be divided up and assigned to the pursuit of stated 
educational goals. 

A D An evaluation of an educar^n^l -rograni should include a 

critical analysis of the Vu'i,e c.r the goals of the program. 

A D Every teacher should have formal ways of gathering informa- 
tion about the strengths and shortcomings of his instruc- 
tional program. 

A D Money spent on evaluation contributes more to the improve- 
ment of education than any other expenditure. 

A D There just is no way that careful and honest evaluation can 
hurt a school program. 

A D If an evaluation study is well designed, the primary find- 
ings are likely to improve decisions made by administrators, 
teachers, and students themselves. 

A U When the evaluator has to choose between helping his staff 
run its program better and helping educators everywhere 
understand all programs a little better he should choose 
the latter. 



Siak(i, '\iobGrl C[irbiina, Illinois: C'>ntcr for Instructional _ 

Uer>oarrh and Curriculum Lvaluution, University of Illinois, ly/i 
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CIRCE Attitude Scale No. 1 .4b 

Scoring of attitudes toward educational evaluation should be made follow- 
ing completion of CIRCE Attitude Scale. 

Different people have different ideas about the evaluation of education- 
al programs. Some believe that maintaining a good school and improving 
instruction require carefully planned evaluation. Others believe that 
evaluation activities interfere with teaching and learning, doing more 
harm than good. 

Different people see different purposes for educational evaluation. Cer- 
tain people are oriented more to pupil behaviors or to classroom condi- 
tions or to other aspects of the program. 

Responses to the items on this attitude scale provide us with 6 scale 
scores. When plotted on the profile sheet below they are expected to 
indicate the respondent's attitudes toward educational evaluation. 



pi recti ons for Sejf Scoring 

Start in the opposite corner of the page. For each scale check your 
sheet to see how you responded to each of the eleven items. For exam- 
ple, with SCALE V how did you mark Item //2? If you marked it "A" put a 
check in the parentheses. Put the number of checks in the box. Mark 
each horizontal scale (at the right) at the numberpoint shown in its 
box. Draw your profile by connecting your scores on the five scales, 
I-V. Then find your CONFIDENCE score by using the formula below. 

To Obtain an overall CONFIDENCE IN EVALUATION 3 

score, do the same thing wit:h tlie check-list 4:^4^4^ 4^4^ojf>o 
a t the ri (jh t . 00 --j cn ai o -p* co cn ro co 

>>> >o>> cz?>oo 

01'^ 3 4 5 G 7 f5 9 10 11 ^^^.^ ^..^^^ 
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□ 



I. A RESEARCH orientation to Evaluation 



<:q<:<: ci <c ci <c ^x. a ^x. 13 



01 2 3 4 5 6 __^_8_1_10_11_ 



S fo^^«D o-~ro 000.0. o The person high on this scale appears 

(/, ^csjc^ rorororo roro^ H- bel ieve that evaluation should rely 

on precise measurement and statistical 
analysis to gain general understanding 
of why programs do or do not succeed. 



□ 



II. A SERVICE orientation to Evaluation 



01 2 3 4 5 6 7 8 9 lOJl 



< ^-ztKocn ^^^^ R ?o ^ ° The person high o.i this scale appears 

CO 



□ 



to believe that evaluatiin she ild be 
designed according to the needs of the 
educators involved so as to aid them 
in their present work and future de- 
cisions. 

III. A TEACHING orientation to Evaluation 

OjTjT__5 6 7_8 9 10 11 

The person high on this scale appears 



<<<Q <<<< Oh <C <C 2 

^o^r^o ° to'believe that evaluation should be 



O 

CO 



□ 



focused considerably on the quality 
of teaching and should discover the 
intrinsic merit in facilities and in 
instruction. 

IV. OBJECTIV- ■ > tion to Evaluation 



03 



< r^o^o^ ir>r^a>^ r^^ro o i . .;or. Ti ' ^ on this scale appears 



01 2 3 4 -> ' _ 7 8 9 10 1 1 

_ _ ^, _ . O 

^ ■ — -^A ^^.-c-o cw^ h- to'be .ovc M:' . instruction, and' there 

fore P/':. lu,.' i, lo , should b,' focused con- 
sider., i.n . '• iori statements of ob- 
jecti '".. ' Che merit of the program 
is lai' 'I, .Jicated by the success uf 
studer . reaching those objectives. 



D 



_j c-j: '0- <C Q -a: -X o Ci X < -X fO 



A JUDGMIi'i-iT orientation to i-wr' I'jation 
01 2 3 4 5 6 7 3 9 _ i .O .; i 



t'-oci r- o: - -^J-o^-* o j,-,g person high on this scci^' joperv-^s 

^ cj.j.MO rviro^ to believe that educational evaluation U 

Ii5 is largely a matter - r'stabl ishing the 



IS largely d iiidLLur r-a\,aui 10.. ...vj ^..^ , ■ 
worth of the prograir fo. various purposes-^J^ 
as poiceived by vario.r groups of per- ^ [) 
sons in and around the program. 
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